High performance computing (HPC) systems are capable of delivering sustained performance approaching 1015 floating point operations per second (petaflops) on real applications; they work with very large data sets and/or they consume large amounts of memory. The HPC systems used today employ multitudes of processors, such as the Blue Gene/L, designed by IBM in conjunction with Lawrence Livermore National Laboratory and ranked as the world's fastest supercomputer in October of 2005 with 131,072 processors. The Blue Gene/L [Adiga N. R. et. al. 2002. “An Overview of the BlueGene/L Supercomputer.” In Proc. IEEE/ACM SC02. Baltimore] can sustain 280.6 trillion calculations per second, called 280.6 teraflops. Systems need upwards of 100K processors to achieve petascale performance. These systems comprise an interconnect system that connects the system processors.
Large-scale HPC systems rely on interconnects and interconnects affect the cost and performance of these systems. The increasingly high cost of high-bandwidth electronic interconnects is due to the expensive optical transceivers needed between switches.
HPC systems use packet-switched networks to interconnect system processors. Inter-processor messages are broken into packets. These packets are then routed through network switches. As system size scales up, a scalable interconnect can consume a disproportionately high portion of the system cost when striving to increase bandwidth while reducing latency. The high cost of such systems has created a need for cheaper alternatives to fulfill the needs of large-scale applications.
The NEC Earth Simulator is an example of a circuit switching based network. The Earth Simulator (ES) [Habata, S, Umezawa, K., Yokokawa, M., and Kitawaki, S. 2004. “Hardware system of the Earth Simulator,” Parallel Computing, 30(12), pp. 1287-1313] network uses a huge electronic crossbar, with 640×640 ports.
The ICN (Interconnection Cache Network) [Gupta, V. and Schenfeld, E. 1994, “Combining Message Switching with Circuit Switching in the Interconnection Cached Multiprocessor Network” in Proc. IEEE Int. Symposium on Parallel Architectures, Algorithms and Networks—Ispan Horiguchi, S. (ed.), pp. 143-150] is another example of a network that comprises processing nodes grouped into small clusters. A drawback of the ICN is that the ratio of the number of circuits to the number of processors in a node is limited to one-to-one.
A drawback of the ES is that although an Electronic Circuit Switch (ECS) has fast circuit setup and release times, for some AMR codes and other irregular communication patterns with switching degree needs larger than 8 circuits from each node, the ES may perform poorly.
Another prior approach is the Gemini system [see Chamberlain, R, Franklin, M., and Baw, C. S., “Gemini: An optical interconnection network for parallel processing,” IEEE Transactions on Parallel and Distributed Processing, 13(10), pp. 1038-1055 (2002)] which comprises a dual multistage network wherein each node has only one optical port and one electrical port into the dual network structure. The Gemini system has the drawback that while it is advantageous to use an optical circuit for long messages that need to be transferred to only one specific destination, HPC applications require more than one destination per node.
In the Clint system [Eberle, H. and Nilsm Gura N., “Separated High-bandwidth and Low-latency Communication in the Cluster Interconnect Clint.” Proceedings of the IEEE/ACM Supercomputing Conference, Baltimore (2002)], as in the Gemini system, there is no sharing of already established circuits among different nodes. Each node sets up its own circuit and may have enough data to send through this circuit before it needs to tear it down and setup another circuit to another destination. Both the Gemini and Clint systems have the shortcoming that a node can communicate with other nodes in the system in a limited manner.
There have been attempts to use the Internet as the network between large processing sites. In such systems the computing is performed over large distances communicates through optical fibers, with potentially different TCP/IP protocols. Two examples of such attempts are the Cheetah [Veeraraghavan, M., Zhenga, X., Leeb, H., Gardnerc, M., and Fengc, M. “CHEETAH: circuit-switched high-speed end-to-end transport architecture,” Proceedings of the SPIE, Volume 5285, pp. 214-225 (2003] and OptiPuter [Defanti, T., Brown, M., Leigh, J., Yu, O, He, E, Mambretti, J., Lillethun, D., and Weinberger, J., “Optical switching middleware for the OptiPuter,” IEICE Transact. Commun. E86-B, 8, pp. 2263-2272 (2003)] projects. However, these systems have the drawback that neither allows the running of fine-grained HPC applications.
Therefore there is a need for a computer switching network that overcomes the above shortcomings of the prior art.